Papers by Pedro Henrique Luz De Araujo
Functionality learning through specification instructions (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Creating or annotating instances targeting specific functionalities is costly and further training models is expensive. |
| Approach: | They propose to use specification instructions to create specification-augmented prompts for each functionality in a suite and combine them with language models pre-trained on natural instruction data. |
| Outcome: | The proposed test suites can assess models’ performance on specific functionalities on four tasks and models of diverse sizes and families. |
Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Styles (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies neglect to measure the generalization of their methods to other prompt styles and different sizes of LLMs. |
| Approach: | They propose a framework that trains an auxiliary model for confidence estimation that aggregates responses from multiple LLMs to capture inter-model agreement. |
| Outcome: | The proposed framework integrates response agreement and focal loss with binary cross-entropy to improve calibration from baselines. |